Parameters | deg. of freedom (real) scale matrix ( pos. def) |
---|---|
Support | positive definite matrices |
Mean | |
Mode | |
Variance | |
CF |
In statistics, the Wishart distribution is a generalization to multiple dimensions of the chi-squared distribution, or, in the case of non-integer degrees of freedom, of the gamma distribution. It is named in honor of John Wishart, who first formulated the distribution in 1928.[1]
It is any of a family of probability distributions defined over symmetric, nonnegative-definite matrix-valued random variables (“random matrices”). These distributions are of great importance in the estimation of covariance matrices in multivariate statistics. In Bayesian inference, the Wishart distribution is of particular importance, as it is the conjugate prior of the inverse of the covariance matrix (the precision matrix) of a multivariate normal distribution.
Contents |
Suppose X is an n × p matrix, each row of which is independently drawn from a p-variate normal distribution with zero mean:
Then the Wishart distribution is the probability distribution of the p×p random matrix
known as the scatter matrix. One indicates that S has that probability distribution by writing
The positive integer n is the number of degrees of freedom. Sometimes this is written W(V, p, n). For n ≥ p the matrix S is invertible with probability 1 if V is invertible.
If p = 1 and V = 1 then this distribution is a chi-squared distribution with n degrees of freedom.
The Wishart distribution arises as the distribution of the sample covariance matrix for a sample from a multivariate normal distribution. It occurs frequently in likelihood-ratio tests in multivariate statistical analysis. It also arises in the spectral theory of random matrices and in multidimensional Bayesian analysis.
The Wishart distribution can be characterized by its probability density function, as follows.
Let W be a p × p symmetric matrix of random variables that is positive definite. Let V be a (fixed) positive definite matrix of size p × p.
Then, if n ≥ p, W has a Wishart distribution with n degrees of freedom if it has a probability density function given by
where Γp(·) is the multivariate gamma function defined as
In fact the above definition can be extended to any real n > p − 1. If n ≤ p − 2, then the Wishart no longer has a density—instead it represents a singular distribution. [2]
The characteristic function of the Wishart distribution is
In other words,
where denotes expectation. (Here and are matrices the same size as ( is the identity matrix); and is the square root of −1).
If has a Wishart distribution with m degrees of freedom and variance matrix —write —and is a q × p matrix of rank q, then
If is a nonzero constant vector, then .
In this case, is the chi-squared distribution and (note that is a constant; it is positive because is positive definite).
Consider the case where (that is, the jth element is one and all others zero). Then corollary 1 above shows that
gives the marginal distribution of each of the elements on the matrix's diagonal.
Noted statistician George Seber points out that the Wishart distribution is not called the “multivariate chi-squared distribution” because the marginal distribution of the off-diagonal elements is not chi-squared. Seber prefers to reserve the term multivariate for the case when all univariate marginals belong to the same family.
The Wishart distribution is the sampling distribution of the maximum-likelihood estimator (MLE) of the covariance matrix of a multivariate normal distribution with zero means. The derivation of the MLE involves the spectral theorem.
The Bartlett decomposition of a matrix W from a p-variate Wishart distribution with scale matrix V and n degrees of freedom is the factorization:
where L is the Cholesky decomposition of V, and:
where and independently. This provides a useful method for obtaining random samples from a Wishart distribution.[3]
It can be shown [4] that the Wishart distribution can be defined if and only if the shape parameter n belongs to the set
This set is named after Gindikin, who introduced it[5] in the seventies in the context of gamma distributions on homogeneous cones. However, for the new parameters in the discrete spectrum of the Gindikin ensemble, namely,
the corresponding Wishart distribution has no Lebesgue density.